Add baseline glutton by maxsmythe · Pull Request #243 · agent-substrate/substrate

Max Smythe (maxsmythe) · 2026-06-13T02:54:13Z

Adds the glutton image to support easier load testing contemplated in #98

This is a rough implementation to help bootstrap more repeatable testing and is expected to change over time as we learn more about performance/testing requirements.

Eitan Yarmush (EItanya) · 2026-06-15T15:15:09Z

Adds the glutton image

What does this mean?

Can you please add a longer PR description so it's clearer to potential reviewers what's happening here?

Max Smythe (maxsmythe) · 2026-06-15T20:45:51Z

Sorry, linked the issue to the PR, but not the PR to the issue. Updated description.

Eitan Yarmush (EItanya) · 2026-06-15T21:01:25Z

Sorry, linked the issue to the PR, but not the PR to the issue. Updated description.

Thanks :)

Bowei Du (bowei)

Generally lgtm. You might want to document in the RPC defs what each glutton action intends to do.

Bowei Du (bowei) · 2026-06-17T19:19:23Z

+	}
+}
+
+func (s *gluttonService) WriteRAM(ctx context.Context, req *glutton.WriteRAMRequest) (*glutton.WriteRAMResponse, error) {


I don't think this makes a huge difference, but this seems to do more than what I think of for writeRAM as it allocates 2x the RAM (if the argument Size is like 32 Gb or something).

Overwrite should just touch bytes but do not extra allocation?

This is a good point. Writing out a larger array will still require swapping the buffer, but we can at least deal with writing to subsets of the buffer.

My main hope with this one is to give us the ability to more easily see the effect of different snapshotting systems.

would be helpful to summarize behavior in a 1-2 sentences for the WriteRAM docstring.

Max Smythe (maxsmythe) · 2026-06-17T19:48:29Z

Thanks for the review! addressed comments and rebased.

Max Smythe (maxsmythe) · 2026-06-17T19:49:03Z

Missed the documentation in the protobuf comment, will integrate into automation PR

Bowei Du (bowei)

Do we want to add typing to the Python code? I know Python types are optional, but it does quickly become a maintenance issue.

Bowei Du (bowei) · 2026-06-22T06:07:52Z

+    def on_start(self):
+        update_user_count(1, self.__class__.__name__)
+
+        target = self.api_host.replace("http://", "").replace("https://", "")


this probably deserves a comment on what this is for

Bowei Du (bowei) · 2026-06-22T06:09:34Z

+        self.api_channel.close()
+
+    def _teardown_actor(self):
+        try:


do we need to track any of these teardown failures?

in later PRs this gets traced, also this code moves to a boomer worker (golang implementation of Locust protocol), probably not worth fixing.

Bowei Du (bowei) · 2026-06-22T06:10:07Z

+                    glutton_pb2.PingRequest(message=msg),
+                    metadata=metadata,
+                )
+                duration = (time.time() - start_time) * 1000


durationMs?

Bowei Du (bowei) · 2026-06-22T06:12:19Z

+spec:
+  runsc:
+    amd64:
+      url: "gs://gvisor/releases/nightly/2026-05-19/x86_64/runsc"


might want a comment here as to where this reference comes from?

This goes away upon rebase (that parameter has been removed)

Bowei Du (bowei) · 2026-06-22T06:16:43Z

+	defer serverboot.ShutdownProvider("MeterProvider", mp.Shutdown)
+
+	if err := os.MkdirAll(*dataDir, 0o700); err != nil {
+		serverboot.Fatal(ctx, "Failed to create data directory", err)


add the dataDir to the error

Bowei Du (bowei) · 2026-06-22T06:17:14Z

+
+	lis, err := net.Listen("tcp", *listenAddr)
+	if err != nil {
+		serverboot.Fatal(ctx, "Failed to start listener", err)


Is the listenAddr in the error (for debugging)

Bowei Du (bowei) · 2026-06-22T06:20:45Z

+	fds   []*os.File
+	peers map[string]*peerGossip
+
+	ramWriteBytes  metric.Int64Counter


Do you think it would be easier to maintain if we should split off each of these kind of operations into different structs with a abstract interface? then we wouldn't have to throw all of the operation code into this one struct.

something like

type action interface { register() do() stop() }

personal preference to also split this into different files to avoid this become a mega file.

the other thing you will want to do is to keep the package main to a minimum and put the real logic into a testable package. You can't add unit tests to the main package.

Move the code to cmd/testing/glutton/internal...

I think some refactoring makes sense, but I don't want to do it in this specific PR because other PRs are already built on top of it and splitting files starts to get hairy WRT rebasing. Happy to tackle this in a follow-on PR.

Bowei Du (bowei) · 2026-06-22T06:24:39Z

+	}
+}
+
+func (s *gluttonService) WriteRAM(ctx context.Context, req *glutton.WriteRAMRequest) (*glutton.WriteRAMResponse, error) {


would be helpful to summarize behavior in a 1-2 sentences for the WriteRAM docstring.

Max Smythe (maxsmythe) · 2026-06-22T18:50:14Z

Added doc strings.

Addressed all comments except ones that will be mooted by follow-on PRs and deferring glutton refactor to avoid rebase difficulty.

Bowei Du (bowei) · 2026-06-22T20:47:19Z

/lgtm

Max Smythe (maxsmythe) mentioned this pull request Jun 13, 2026

Easy Stress Testing #98

Open

Max Smythe (maxsmythe) force-pushed the glutton branch from 8097647 to d2290e4 Compare June 13, 2026 02:59

Max Smythe (maxsmythe) requested a review from Bowei Du (bowei) June 17, 2026 06:37

Max Smythe (maxsmythe) force-pushed the glutton branch from d2290e4 to d0de2a9 Compare June 17, 2026 06:51

Bowei Du (bowei) reviewed Jun 17, 2026

View reviewed changes

Bowei Du (bowei) self-assigned this Jun 17, 2026

Max Smythe (maxsmythe) force-pushed the glutton branch from dd6ff79 to 1947706 Compare June 17, 2026 19:48

Max Smythe (maxsmythe) requested a review from Bowei Du (bowei) June 17, 2026 19:48

Max Smythe (maxsmythe) mentioned this pull request Jun 18, 2026

Add Benchmarking Orchestration #275

Open

Bowei Du (bowei) reviewed Jun 22, 2026

View reviewed changes

Max Smythe (maxsmythe) added 2 commits June 22, 2026 18:48

Add baseline glutton

fdbc2d2

Fix imports; overwrite RAM buffer; add status metric

2120f31

Max Smythe (maxsmythe) force-pushed the glutton branch from 1947706 to f2f8587 Compare June 22, 2026 18:49

Add Python type headers + general cleanup

b0aa689

Max Smythe (maxsmythe) force-pushed the glutton branch from f2f8587 to b0aa689 Compare June 22, 2026 18:52

Max Smythe (maxsmythe) requested a review from Bowei Du (bowei) June 22, 2026 19:15

Bowei Du (bowei) approved these changes Jun 22, 2026

View reviewed changes

Bowei Du (bowei) merged commit 3a6660e into agent-substrate:main Jun 22, 2026
9 checks passed

Uh oh!

Conversation

Max Smythe (maxsmythe) commented Jun 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Eitan Yarmush (EItanya) commented Jun 15, 2026

Uh oh!

Max Smythe (maxsmythe) commented Jun 15, 2026

Uh oh!

Eitan Yarmush (EItanya) commented Jun 15, 2026

Uh oh!

Bowei Du (bowei) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Max Smythe (maxsmythe) commented Jun 17, 2026

Uh oh!

Max Smythe (maxsmythe) commented Jun 17, 2026

Uh oh!

Bowei Du (bowei) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Max Smythe (maxsmythe) commented Jun 22, 2026

Uh oh!

Bowei Du (bowei) commented Jun 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Max Smythe (maxsmythe) commented Jun 13, 2026 •

edited

Loading